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Operational  Need 


Sensemaking 


Cognitive 

Domain 


Situation  Awareness  is  Critical  to  C2  performance,  Fatigue  Effects  Performance 

-  How  does  it  Affect  SA 


Fatigue  Effects  Performance 


Effects  of  fatigue  on 
performance  well  recognized 

Most  studies  have  been 
conducted  with  pilots  and 
drivers 

Understood  that  fatigue  effects 
cognitive  performance,  not  well 
understood  how  that  is 
manifested  in  C2  tasks 

Some  initial  studies  show  likely 
impact,  situation  awareness  has 
not  been  specifically  addressed 


Hours  Without  Sleep 

10  12  14  16  18  20  22  24  26 


Fatigue  Effects  Performance,  cont’d 


■  Effects  of  chronic  fatigue 
similar  to  sleep 
deprivation 

■  Participants  not  aware  of 
deprivation 

■  Understood  that  fatigue 
effects  cognitive 
performance,  not  well 
understood  how  that  is 
manifested  in  C2  tasks 

■  Some  initial  studies 
show  likely  impact, 
situation  awareness  has 
not  been  specifically 
addressed 


digit  symbol  substitution  task  serial  addition/subtraction  task 


Hans  P.A.  Van  Dongen,  Greg  Maislin,  Janet  M.  Mullington,  David  F.  Dinges,  2004 
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Fatigue  Management  in  C2 


■  Fatigue  models 

■  Cognitive  assessment 
and  models 

■  Physiological  sensors 
and  model 

■  Countermeasures 
(system  adaptation, 
breaks,  scheduling, 
etc...) 


Fatigue  detection  system  and  countermeasures  that 
prevent  degradation  of  operational  performance 
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Mapping  Stressors  to  Cognition  -  Impact  of  Fatigue 
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Table  from  Boeing  presentation  at  1^^  Annual  Augmented  Cognition  Conference 


New  Discipline  -  Augmented  Cognition 


■  Goal  -  enhance  operational  effectiveness  by  deveioping 
technologies  capable  of  extending  the  information 
management  capacity  of  the  warfighter  via  closed  loop 
system  that  adapts  to  the  state  of  the  warfighter  to 
significantly  improve  performance. 


■  Technologies 

■  Physiological  Sensors 

■  Cognitive  State  Assessors 

■  Cognitive  Models 

■  System  Automation/Scheduling  tools 


API 
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Physiological  Measures  to  Assess 
Cognitive  State 


Heart  Rate 

Heart  Rate  Variability 

Activity 

Eye  Scanning 

PERCLOS 
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Research  Need 


■  Significant  research  in  the  academic  iab 

■  Need  to  extend  to  the  warfighter 

■  Assess  with  operationai  (or 
simuiations  of  actuai  systems) 

■  Empioy  operationaiiy  reaiistic 
scenarios 

■  Test  with  representative  sampies  - 
actuai  warfighters 
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Near  Term  Research  Objectives 


■  Demonstrate  measurement  of  SA  over 
extended  durations 

■  Demonstrate  the  reiationship  of  fatigue 
and  situation  awareness 

■Determine  if  SA  (high,  medium,  low) 
degrades  differentially  over  time 

■Validate  Physiological  Measurement  of 
C2  task  performance 


^PL 
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HSI  Technology  Perception  &  Cognition 


Situation  Awareness 


2.  Comprehend 

what  is  changing 
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4.  Anticipate  wha 

will  happen  as  the 
situation  develops 


3.  Assess  how 

the  changes  impact 
the  present  situation 


Perception  &  Cognition 


FY  05  Experiment 


Pilot:  proof  of  process  and  principle 

Investigating  effects  of  fatigue  on  Situation  Awareness 

Matured  theory  of  SA,  SA  assessment,  and  SA  analysis  (patent  pending) 

Investigate  application  of  physioiogicai  measures  for  cognitive  task 
performance  assessment 
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Method 


Briefing  Datts 


Six  JHU/APL  interns  were 
participants 

Ran  2  experimental  sessions 

■  Training  session  ~16  hours  prior 
to  experiment 

■  36-hour  duration 

■  Wake-up  call  0600  Fri, 

■  baseline  0800  Fri, 

■  testing  1800  Fri  thru  1800  Sat 

Collected  PVT,  ANAM,  SA 
performance  while  monitoring  hea 
rate,  heart  rate  variability,  and 
activity 


ECG 


■  Tested  every  three  hours  starting 
at  12  hours  of  wakefulness 


Monitored  participant  activity 
between  test  sessions 
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Initial  Findings 


■  Increase  in  RT  during  cycie  5,  (24 
hours  of  wakefulness). 
Corresponding  to  the  highest 
sleepiness  indication  reported  by 
participants. 

■  SA  did  not  decline  significantly, 
but  did  not  spike  as  couid  be 
expected  due  to  learning  curve. 

"  Microwave  activity  sensor 
well  suited  to  detect  sleep 
onset,  not  shown  to  be 
effective  in  this  type  of 
physical  configuration. 
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Results:  Summary 


■  Overall  SA  might  be  a  sufficient  singie  summary  metric  of  SA 

■  SA  Levei  1  and  Levei  2  showed  correiation  with  Overaii  SA  and  with  each 
other 

"  SA  Levei  3  did  not  correlate  with  any  of  the  others 

■  This  is  not  surprising,  as  the  participants  were  novice  users.  The 
ANAM  and  PVT  Sieepy  correiations  with  SA  indicate  that  as  a 
participant’s  self-reported  sleepiness  increases,  SA  ieveis  decreased 


"  Showed  Potentiai  Reiationship  Between  Fatigue  and  Physioiogicai  Measures 

■  Consistent  with  iiterature,  PVT  Deiay  and  PVT  Sieepy  correlated  strongly; 
indicates  fatigue 

"  PVT  measures  had  strong  positive  correiation  with  Heart  Rate  Variability; 

"  PVT  had  negative  correlations  with  HR,  ANAM  Logic,  and  ANAM  Code 
expected;  more  tired,  siower  response,  &  less  capable  of  solving  logic  & 
math  problems 

"  The  decreased  HR  correlated  to  an  increase  in  ANAM  Sleepy,  but  also  to 
an  increase  in  ANAM  Math.  As  noted  on  the  previous  slide,  this  increase 
in  math  abiiities  as  participants  become  more  tired  is  interesting 

■  The  self-reported  ANAM  Sleepy  provided  an  indication  of  participant’s 
performance  on  ANAM  Logic  and  ANAM  Code,  as  sieepiness  increased 
scores  decreased 


Correlation  Coefficients  between  SA  Levels 


Overall 

SA 

SA 

Level  1 

SA 

Level  2 

SA 

Level  3 

OSA 

0.814 

0.921 

0.359 

SA1 

0.579 

0.369 

SA2 

0.278 

SA3 

Correlation 
stronger 
than  0.5 


■  SA  Level  1  and  Level  2  showed  correlation  with  Overail  SA  and  with 
each  other 

■  SA  Level  3  did  not  correlate  with  any  of  the  others 

■  This  is  not  surprising,  as  the  participants  were  novice  users. 
They  likely  did  not  possess  the  knowledge  of  the  subject  area  or 
system  to  make  projections  about  future  trends 
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Overall  SA  might  be  a  sufficient 
single  summary  metric  of  SA 


Correlation  Coefficients  between  SA  and  other  Tests 


PVT 

Average 

Delay 

PVT 

Sleepy 

HR 

HRV 

ANAM 

Sleepy 

ANAM 

Math 

ANAM 

Logic 

ANAM 

Code 

OSA 

-0.165 

-0.514 

0.193 

0.006 

-0.560 

0.325 

0.124 

0.338 

SA1 

0.181 

-0.275 

0.020 

0.204 

-0.320 

-0.037 

-0.280 

0.021 

SA2 

-0.255 

-0.492 

0.199 

-0.102 

-0.620 

0.567 

0.228 

0.499 

SA3 

-0.254 

-0.446 

0.512 

-0.143 

-0.201 

0.333 

0.610 

-0.279 

■  The  ANAM  and  PVT  Sleepy  correlations  with  SA  indicate  that  as  a 
participant’s  self-reported  sleepiness  Increases,  SA  levels  decreased 

■  The  SA  3  and  ANAM  Logic  correlation  indicate  that  participants  who 
were  better  at  iogic  were  also  better  able  to  make  the  predictions 
(necessary  for  Level  3) 
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Preliminary  Evidence  to  Support 
Fatigue  Management  Concept 


Correlation  Coefficients  between  PVT  and  other  Tests 


PVT 

Average 

Delay 

PVT 

Sleepy 

HR 

HRV 

ANAM 

Sleepy 

ANAM 

Math 

ANAM 

Logic 

ANAM 

Code 

PVT 

Average 

Delay 

0.773 

-0.700 

0.693 

0.750 

0.168 

-0.818 

-0.614 

PVT 

Sleepy 

-0.888 

0.720 

0.953 

0.580 

-0.729 

-0.661 

■  Consistent  with  literature,  PVT  Delay  and  PVT  Sleepy  correlated  strongly; 
indicates  fatigue 


"  The  PVT  Sleepy  correlated  significantly  with  ANAM  Sleepy  as  expected 

"  Both  PVT  measures  had  strong  positive  correlation  with  Heart  Rate 
Variability;  PVT  Sleepy  also  correlated  with  ANAM  Math,  an  oddity  that 
warrants  further  investigation 

■  Negative  correlations  with  HR,  ANAM  Logic,  and  ANAM  Code  expected;  more 
tired,  slower  response,  &  less  capable  of  solving  logic  &  math  problems 
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Shows  Potential  Relationship  Between 
Fatigue  and  Physiological  Measures 


Correlation  Coefficients  between  HR  and  other  Tests 


HR 

HRV 

ANAM 

Sleepy 

ANAM 

Math 

ANAM 

Logic 

ANAM 

Code 

HR 

-0.744 

-0.785 

-0.554 

0.640 

0.369 

HRV 

0.752 

0.311 

-0.717 

-0.535 

■  Heart  Rate  and  Heart  Rate  Variability  had  a  strong  negative 
correlation 

■  The  decreased  HR  correlated  to  an  increase  in  ANAM  Sleepy,  but 
also  to  an  increase  in  ANAM  Math.  As  noted  on  the  previous  slide, 
this  increase  in  math  abilities  as  participants  become  more  tired  is 
interesting 

■  As  HR  increased,  so  did  ANAM  Logic  scores;  however,  they  were 
measured  several  minutes  apart,  not  simultaneously,  so  it  is 
difficult  to  draw  conclusions 


Shows  Potential  Relationship  Between 
Fatigue  and  Physiological  Measures 
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Correlation  Coefficients  within  AN  AM  Tests 


ANAM 

Sleepy 

ANAM 

Math 

ANAM 

Logic 

ANAM 

Code 

ANAM  Sleepy 

0.487 

-0.641 

-0.817 

ANAM  Math 

-0.147 

-0.319 

ANAM  Logic 

0.460 

ANAM  Code 

■  The  self-reported  ANAM  Sleepy  provided  an  indication  of 
participant’s  performance  on  ANAM  Logic  and  ANAM  Code,  as 
scores  decreased  as  sieepiness  increased 
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Preliminary  Evidence  that  Cognitive 
Capabilities  Degrade  with  Fatigue 


FY05  Research  Issues 


■  Did  we  really  evaluate  SA? 

■  What  other  physiological  measures  can  we 

■  Likely  a  composite  is  more  powerful  than 
single  metric. 

■  Does  SA  degrade  differently  if  different 
quality  is  achieved? 

■  Does  poor  SA  degrade  more/less  than  good 
SA? 


FY06  Experiment 


"  Add  eye-tracker  (gaze  patterns, 
saccades,  PERCLOSE) 
measures 

■  Test  experts  to  fully  gauge  SA 

■  Test  two  instantiations  of  a 
system,  compare  SA,  evaluate 
if/how  SA  degrades  differentially 
over  time 


■  Run  study  late  spring/early 
summer 


Visual  &  3D 


visual 


Wireless  head-tracker 
synchs  video  display 
with  spatial  audio 


*  Full-sensor  monitoring 

•  Single  or  multiple  D/E’s 
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Summary: 

FY05  experiment  provides  indication  that 
approach  and  measures  are  useful  for 
assessing  fatigue  effects 

SA  appears  to  be  affected  by  fatigue 

FY06  help  to  solidify  findings  and  provide 
foundation  for  C2  fatigue  management 


